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Editor's key points 

. The World Health 
Organization's Surgical 
Safety Checklist was 
introduced to improve 
perioperative morbidity, 
mortality, and adherence 
to clinical protocols. 

• The role of changes in 
safety culture in the 
positive effects of this 
checklist was assessed in a 
prospective controlled 
intervention survey in 
operating theatre 
personnel. 

• Successful checklist 
implementation had 
limited impact on patient 
safety culture in this 
single-site study, for 
unclear reasons that 
require further study. 



Background. Positive changes in safety culture have been hypothesized to be one of the 
mechanisms behind the reduction in mortality and morbidity after the introduction of 
the World Health Organization's Surgical Safety Checklist (SSC). We aimed to study the 
checklist effects on safety culture perceptions in operating theatre personnel using a 
prospective controlled intervention design at a single Norwegian university hospital. 

Methods. We conducted a study with pre- and post-intervention surveys using the 
intervention and control groups. The primary outcome was the effects of the Norwegian 
version of the SSC on safety culture perceptions. Safety culture was measured using the 
validated Norwegian version of the Hospital Survey on Patient Safety Culture. 
Descriptive characteristics of operating theatre personnel and checklist compliance data 
were also recorded. A mixed linear regression model was used to assess changes in 
safety culture. 

Results. The response rate was 61% (349/575) at baseline and 51% (292/569) post- 
intervention. Checklist compliance ranged from 77% to 85%. We found significant 
positive changes in the checklist intervention group for the culture factors 'frequency of 
events reported' and 'adequate staffing' with regression coefficients at -0.25 [95% 
confidence interval (CI), -0.47 to -0.07] and 0.21 (95% CI, 0.07-0.35), respectively. 
Overall, the intervention group reported significantly more positive culture scores- 
including at baseline. 

Conclusions. Implementation of the SSC had rather limited impact on the safety culture 
within this hospital. 
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An estimated 234 million major surgical operations are per- 
formed annually worldwide. 1 As volume and importance of 
surgery in global healthcare increase, patient safety and 
quality in surgical care gain more attention. 2 3 Nearly one 
in 10 in-hospital patients experience iatrogenic events and 
more than half of them occur within perioperative care. 4 

In 2008, the World Health Organization (WHO) launched 
the Safe Surgery Saves Lives campaign and produced the 
'Surgical Safety Checklist' (SSC) designed to reduce 



complications and deaths associated with surgery. 5 In an 
international pilot study, the SSC intervention resulted in a 
decrease in mortality (1.5-0.8%) and morbidity (17-11%). 6 
Similar effects were found after implementing the more 
comprehensive Surgical Patient Safety System (SURPASS) 
checklist on patient outcomes in the Netherlands. 7 An import- 
ant purpose of introducing the WHO SSC was to improve basic 
clinical processes as shown by the increase in appropriate 
antibiotic use from 56% to 83%, correct site marking from 
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54% to 92%, and overall clinical safety processes from 34% to 
57%, suggesting improved reliability in clinical care. 6 

Within the healthcare and other industries, checklists 
are more than a simple intervention. At a basic level, they 
function as reminders, which ensure basic care processes 
are adhered to (assuming whichever checklist is in place is 
used correctly). At a broader level, checklists and their 
usage have implications for team working, team cohesion, 
and safety culture. Checklists require people to change 
their work routines — for example, the Time Out phase of 
the WHO SSC requires the entire operating theatre team to 
gather and pause for a few seconds before proceeding with 
a procedure. Given that the healthcare industry was rarely 
using such interventions until recently, it has been argued 
that checklists are not a panacea that will fix every safety 
problem— rather they are likely to interact with the team 
and safety culture of the local team and wider organization. 8 
If significant wider problems exist within an organization, the 
likely outcome is that a checklist will not have a positive 
benefit, and indeed, it may be reduced to a tick box exercise. 9 

Along these lines, checklist-driven improvements have 
been hypothesized to impact positively on team and safety 
culture and, in turn, to drive decreases in patient mortality 
and morbidity. 6 Safety culture relates to personnel's atti- 
tudes, common thoughts, and behaviours within an organ- 
ization. 10 Although not easy to measure, a number of 
surveys that assess safety culture have been published 11 — 
alongside studies that investigate culture via ethnographic 
approaches and observation. 12 13 Survey instruments typical- 
ly investigate a range of facets of culture, including team 
working, 14 15 communication, 16 17 and attitudes to safety. 18 
Studies to date have linked occurrence of patient safety inci- 
dents with safety culture and hence tools to monitor culture 
within hospitals have been implemented. 11 19 

To date, the effects of the WHO SSC have been evaluated 
regarding compliance, 20 communication, 21 22 staff attitudes, 
and partly safety culture. 18 22 23 Published studies are typic- 
ally pre-/post-implementation designs without control groups. 
The primary aim of this study was to measure the effects of 
the WHO SSC on operating theatre personnel perceptions of 
safety culture using a controlled study design. We hypothesize 
that implementation of the SSC is associated with positive 
changes in safety culture. 

Methods 

The study was reviewed by the Regional Committee for 
Medical and Health Research Ethics (Ref: 2009/561) and the 
hospital privacy Ombudsman, who approved it (Ref: 2010/ 
413). Written informed patient consent was waived. Operat- 
ing theatre personnel gave consent by responding to the 
surveys. 

Study design 

This was a prospective controlled intervention study using 
pre- and post-intervention surveys with the intervention 
and control groups. The primary outcome was the changes 



of safety culture perceptions in operating theatre personnel 
after implementation of the Norwegian version of the 
WHO Surgical Safety Checklist, introduced after WHO 
guidelines. 5 A randomized stepped wedge design 24 was uti- 
lized to determine the order of intervention introduction 
across three surgical specialities (orthopaedic, thoracic, and 
neurosurgery — see the following section for details) in the 
intervention site of the hospital. Compliance with checklist 
usage was the secondary outcome. 

Study population 

The study took place in Haukeland University Hospital, a 
1100-bed tertiary university hospital in the western part of 
Norway. The perioperative setting comprised 10 surgical 
departments and the accompanying departments of anaes- 
thesia and intensive care administering anaesthesia and 
perioperative care. The target population of perioperative 
personnel included all eligible surgeons, anaesthetists, oper- 
ating theatre nurses, nurse anaesthetists, and ancillary per- 
sonnel (unit assistants, clerks, and cleaning assistants) 
located at two separate sites. The intervention group com- 
prised personnel from orthopaedic surgery, thoracic 
surgery, and neurosurgery placed at the central hospital 
site. The control group comprised personnel from ear, nose, 
and throat; maxillofacial; plastic; endocrine; urology; gastro- 
intestinal; obstetric; and gynaecological surgery specialities 
located at the peripheral hospital site. Within the hospital, 
operating theatre clinical and other personnel work in the 
separate sites without rotation, except for a few anaesthe- 
tists. Inclusion was based on work list information. A 
census approach was taken for recruitment — with the entire 
target population (as described above) invited to take part in 
the study. A total of 349 participants responded at baseline 
and 292 responded at post-intervention. 

Study procedure 

The study was carried out over 9 months from October 2009 
to July 2010. Baseline and post-Checklist intervention survey 
data were collected during two 4 week periods in October 
2009 and June 2010 (Fig. 1). The surveys were forwarded 
to the operating theatre personnel using both hospital elec- 
tronic mail and the internal mail system (i.e. hardcopies). 
Identification numbers were assigned to or printed on each 
questionnaire to match individuals for the pre- and post- 
intervention surveys. Compliance with the Checklist was pro- 
spectively recorded (i.e. Checklist 'used' or 'not used') via the 
computer-based operating planning system within the oper- 
ating theatres of the hospital. Nurse anaesthetists and 
theatre nurses also checked manually whether the paper 
versions of the Checklist had been completed for every case. 

Checklist intervention 

The Norwegian version of the SSC was introduced using a 
randomized sequential roll-out of the intervention. 24 In a 
joint venture between the Norwegian National Unit for 
Patient Safety, the Health Trust of Forde, and the Surgical 
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Pre-intervention 

survey 
(A/=575, n = 349) 
October 2009 



Checklist intervention 
8 months time period 
November 2009-June 2010 



Post-intervention 

survey 
(N =569, n = 292) 
June/July 2010 



Control group 
(n = 203) 



tsmsm 




Intervention group 
(n=146) 



Thoracic surgery 



Orthopaedic surgery 



Intervention group 
(n=140) 



Fig 1 Study procedure for the SSC intervention and the pre-intervention and post-intervention surveys at Haukeland University Hospital, 
Bergen, Norway, in 2009-2010. N, subjects included; n, subjects responded. 



Safety Study Group of Bergen, the Checklist was translated 
and adapted to meet Norwegian surgical flow of care. The 
Checklist consisted of 20 items orally confirmed by operating 
theatre personnel aimed at ensuring patient safety during 
anaesthesia and surgery. It was performed at three critical 
junctures in care: before induction of anaesthesia (Sign In), 
immediately before incision or start of treatment (Time 
Out), and before the leading surgeon left the operating 
theatre after surgery (Sign Out). 5 The Sign In part before an- 
aesthesia induction was led by the nurse anaesthetist. The 
Time Out and Sign Out parts were led by the circulating 
nurse. A completed checklist form was included into the 
patient's notes. 

With management leaders support, the Surgical Safety 
Study Group of Bergen introduced the SSC to all special- 
ities/professional groups in the intervention group, using an 
educational programme consisting of lectures, the NHS 
(UK) videos on how to perform and not to perform the 
checklist, 25 information disseminated via e-mails, and WHO 
guideline material 5 translated into Norwegian. The SSC was 
piloted during the two first weeks of implementation result- 
ing in a few minor adjustments — including that the Sign In 
should be led by the nurse anaesthetist with anaesthetist 
and operating theatre nurse present before induction, and 
the Time Out should be performed by the operating theatre 
nurse as the surgeon was ready to start the operation- 
pausing the whole team. Further feedback was received by 
end-users 2 weeks and also 2 months post-initial implemen- 
tation. The randomized stepped wedge implementation 
started with orthopaedic surgery followed by thoracic and 
neurosurgery at 4 week intervals in the intervention site of 
the hospital. 



Outcome measures 

The primary outcome was post-implementation changes in 
safety culture measured by the Norwegian version of the 
Healthcare Research and Quality Hospital Survey on Patient 
Safety Culture (Hospital SOPS). 19 26-28 The survey instrument 
measures hospital staff perceptions of safety culture using 42 
items that cover 12 factors, or elements of culture: 'overall 
patient safety', 'frequency of events reported' (including 
near misses in theatres), 'unit manager/leader promoting 
safety', 'organizational learning— continuous improvement', 
'teamwork within units', 'communication openness', 'feed- 
back on error reported', 'non-punitive response to errors', 'ad- 
equate staffing' (to handle difficult situations in theatre), 
'hospital manager/leader promoting safety', 'teamwork 
across units', and 'quality of information handoffs and transi- 
tions of care'. 28 The first nine factors address culture at clinical 
unit level, whereas the last three factors address culture at the 
wider hospital level. Items are scored on five-point agreement 
scales (1, strongly disagree, to 5, strongly agree) or frequency 
scales (1, never, to 5, always) as appropriate. 19 28 The Hospital 
SOPS instrument had previously not been used within this 
hospital. The instrument was selected based on its very 
good psychometric properties 27 " 29 and also because we 
could compare our findings with previous findings from 
similar populations assessed using the same tool. 27 28 

Statistical analyses 

Statistical analyses were carried out using SPSS version 20 
(SPSS, Chicago, IL, USA). The reliability of the Norwegian Hos- 
pital SOPS instrument in the form of internal consistency was 
assessed using Cronbach's a coefficients. Descriptive 
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performed a x 2 analysis of non-responders to establish pos- 
sible differences with responders regarding gender, groups, 
and profession and found a significant variation (P<0.01) 
for professions in both surveys, with fewer non-responding 
nurse anaesthetists in the pre- and post-intervention 
surveys and more surgeons and ancillary personnel as non- 
responders in the post-intervention survey. Checklist compli- 
ance for the study period was 85% of all cases (elective and 
emergency surgery) for the Sign In, 84% for the Time Out, 
and 77% for the Sign Out (Table 2). 

Norwegian 'Hospital SOPS' reliability 

Reliability was assessed at baseline (n=349) with lowest 
Cronbach's a of 0.60 and 0.64 for 'adequate staffing' and 
'organizational learning and continuous improvement' and 
with the a ranging between 0.67 and 0.85 for the remaining 
factors. At post-intervention (n=292), the lowest a was 0.60 
for 'non-punitive response' and ranged from 0.66 to 0.85 for 
the other factors. Overall, these are acceptable to very good 
levels of reliability for research purposes. 

Checklist intervention effects on safety culture 

Detailed descriptive analyses across the two groups (control 
vs intervention) and the two time-points (pre-intervention 
vs post-intervention) are presented in Table 3. 



Table 1 Characteristics of the intervention (WHO SSC) and control groups (n=641) 





Pre-intervention survey 








Post-intervention survey 






Checklist 


Control 


Total 
n 


P-value 


Checklist 


Control 


Total 
n 


P-value 


n 


% 


n 


% 


n 


% 


n 


% 


Occasion and groups 


146 


41.8 


203 


58.2 


349 




140 


47.9 


152 


52.1 


292 


0.21 


Gender 


























Male 


64 


43.8 


80 


39.4 


144 


0.44 


65 


46.4 


52 


34.2 


117 


0.04 


Female 


82 


56.2 


123 


60.6 


205 




75 


53.6 


100 


65.8 


175 




Profession 












<0.01 












<0.01 


Surgeon 


44 


30.1 


83 


40.9 


127 




32 


22.9 


60 


39.5 


92 




Operating theatre nurse 


35 


24.0 


42 


20.7 


77 




36 


25.7 


35 


23.0 


71 




Anaesthetist 


24 


16.4 


20 


9.9 


44 




31 


22.1 


10 


6.6 


41 




Nurse anaesthetist 


43 


29.5 


33 


16.3 


76 




41 


29.3 


30 


19.7 


71 




Ancillary personnel 






25 


12.3 


25 








17 


11.2 


17 




Patient contact 












0.02 












0.21 


Yes 


140 


98.6 


182 


90.5 


322 




128 


96.2 


140 


92.2 


269 




No 


2 


1.4 


19 


9.5 


21 




5 


3.8 


12 


7.8 


17 




Weekly working hours 


























<20 


2 


1.4 


15 


7.5 


17 




1 


0.7 


5 


3.3 


6 




20-37 


59 


40.7 


80 


39.8 


139 




53 


38.4 


63 


41.9 


116 




>37 


84 


57.9 


106 


52.7 


190 




84 


60.9 


83 


55.0 


167 




Hospital experience (yr) 












0.97 












0.43 


<1 


7 


4.8 


9 


4.5 


16 




9 


6.5 


7 


4.7 


16 




1-5 


34 


23.4 


50 


25.3 


84 




25 


18.1 


42 


28.2 


67 




6-10 


28 


19.3 


36 


18.2 


64 




28 


20.3 


28 


18.8 


56 




11-15 


28 


19.3 


38 


19.2 


66 




26 


18.8 


21 


14.1 


47 




16-20 


20 


13.8 


22 


11.1 


42 




19 


13.8 


17 


11.4 


36 




21-40 


28 


19.3 


43 


21.7 


71 




31 


22.5 


34 


22.8 


65 





statistics quantified sample characteristics and compliance 
data. Each of the 12 patient safety culture factors was based 
on three or four items, which were aggregated to produce a 
mean score for the factor. Negatively worded items were 
reversed to ensure that higher scores overall indicate better 
safety culture. A mean sum score was calculated across all 
12 factors. The intervention and control groups were com- 
pared using analysis of variance (anova). We used a hierarchical 
mixed linear model (MLM) based on multiple regression ana- 
lysis to calculate effects of the SSC intervention. Following sub- 
jects responding both at baseline and post-intervention, the 
MLM test allows for inclusion of subjects responding only at 
baseline or post-intervention. 30 The regression model is 
detailed in the Appendix. Variations between responders and 
non-responders were assessed with the Pearson x 2 test. Stat- 
istical significance was set at two-tailed P<0.05. 

Results 

Sample characteristics 

A total of 641 participants took part in the study. Overall re- 
sponse rates for the two phases of the study were 61% (349/ 
575) at the baseline/pre-intervention survey and 51% (292/ 
569) at the post-intervention survey. Subjects responding in 
both surveys represented 67% (432/641) of the respondents. 
Detailed sample characteristics are displayed in Table 1. We 
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Table 2 Compliance with the WHO SSC in orthopaedic, thoracic, and neurosurgical operations (N=2367) at Haukeland University Hospital, 
Bergen, Norway, in 2009-2010 



Surgery Use of SSC 





N 


Sign In 




Time Out 




Sign Out 




All parts 




n 


% 


n 


% 


n 


% 


n 


% 


Orthopaedic 


1579 


1414 


90 


1386 


88 


1307 


83 


1264 


80 


Thoracic 


393 


337 


86 


338 


86 


300 


76 


287 


73 


Neuro 


395 


264 


67 


257 


65 


225 


57 


216 


55 


All 


2367 


2015 


85 


1981 


84 


1832 


77 


1767 


75 


P-value 




<0.01 




<0.01 




<0.01 




<0.01 





The multivariate analysis with MLM demonstrated a sig- 
nificant effect (P<0.01) of the SSC intervention on the two 
factors 'frequency of events (near misses) reported' and 'ad- 
equate staffing' (Fig. 2). The effect is described by regression 
coefficients for the interaction as -0.25 [95% confidence 
interval (CI), -0.47 to -0.07] and 0.21 (95% CI, 0.07- 
0.35), respectively (Table 4). For instance, for 'frequency of 
events reported', there was an increase of 0.11 from pre to 
post in the control group, but a decrease in the Checklist 
group (0.11-0.25= -0.14). For 'adequate staffing', there is 
hardly any change (b 2 = -0.07) in the control, but an increase 
in the Checklist group (-0.07+0.21=0.14). For the safety 
culture measured on the overall hospital level, the MLM ana- 
lysis also showed a significant effect of the Checklist inter- 
vention for the factors 'hospital management promotes 
safety' and 'quality of information handoffs and transitions 
of care' in both groups. The regression coefficients for the dif- 
ferences were 0.12 (95% CI, 0.04-0.20) and 0.08 (95% CI, 
0.02-0.14), respectively. The same pattern was obtained 
when we adjusted the analyses using profession, gender, 
level of patient contact, and work experience as covariates. 
Subgroup analyses with covariates did not change the 
results. 

Across both baseline and post-intervention, we found sig- 
nificant group differences between the Checklist intervention 
group and the control group, in favour of the intervention 
group, for the factors 'overall patient safety', 'frequency of 
events reported', 'manager promoting safety', 'organizational 
learning-continuous learning', 'teamwork within units', 'feed- 
back/communication about errors', 'non-punitive response to 
error', and the 'sum score' (i.e. overall safety culture scale 
mean score) — details of these differences are presented as 
regression coefficients with 95% CI in Table 4. 

Discussion 

In this prospective controlled intervention study of the WHO 
SSC in Norway, the introduction of the Checklist was asso- 
ciated with rather small impact on patient safety culture 
(measured by the 'Hospital SOPS' scale). Overall, the interven- 
tion group scored higher on a number of baseline culture 
factors— but even taking this into account, we only found 
positive effects on two dimensions of patient safety culture: 
a significant decrease in 'frequency of events reported' and 



a significant improvement in perceptions of 'adequate staff- 
ing' in the Checklist intervention group. The decrease in 
events reported could be associated with a real mitigation 
of near misses or errors after the introduction of the WHO 
Checklist in the intervention group. The SSC effects change 
in theatre routines, such that team members may eventually 
be better prepared for anaesthesia and surgery, hence 
leading to fewer near misses. Improved safety processes in 
the operating theatre have been seen after SSC implementa- 
tion, such as raised awareness in the operating team and 
foreseeing any errors or problems. 6 31 In fact, the SURPASS 
study quantified incidence of errors caught to 40.6% (2562/ 
6313) of checklists, supporting the assumption that check- 
lists prevent near misses and errors. 32 Direct observational 
evidence would be required to further validate this finding. 

The improvement in perceptions of having adequate staff- 
ing to handle difficult situations in theatre is more difficult to 
account for. During the study period, there was no objective 
increase in staffing as an explanatory variable. According to 
data from the hospital personnel system, the number of 
active health personnel was constant during the study 
period and even the numbers on sick leave were not signifi- 
cantly different. It is possible that this effect is entirely sub- 
jective — staff's perceptions of teamwork have been shown 
to be associated with measures of safety and quality in 
patient care, 33 and Bohmer and colleagues 22 found that 
the team introductions during the Time Out contributed to 
improved staff satisfaction. The use of the SSC, including 
team introductions, might have enhanced team cohesion 
and thus affected subjective perceptions of staffing. This 
finding clearly requires further investigation. 

Checklist and safety culture 

Is it possible for operating theatre teams to adopt a practice 
which seems important to them but without broader 
improvements in their attitudes to and perceptions of 
safety? The safety culture factors that did not improve post- 
implementation of the SSC in this study are somewhat differ- 
ent from findings in other studies, especially regarding team- 
work 15 16 and communication. 21 22 3A 

For checklist implementation to be effective, a concur- 
rent cultural change within organizations has been sug- 
gested to be crucial 6 35 36 — indeed, it has been argued 
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Table 3 Descriptive statistics analyses in the intervention (checklist) compared with control groups for the pre-intervention (n=349) and 
post-intervention (n=292) phases of the study, se, standard error; CI, confidence interval 



Safety factors (scale 1-5) 


Pre-intervention 








Post-intervention 






Mean 


SE 


95% 


CI 


Mean 


SE 


95% CI 


Overall patient safety 
















Intervention 


3.63 


0.05 


3.53, 


3.73 


3.69 


0.04 


3.60, 3.77 


Control 


3 C1 
j.j! 


n n/. 


3 /. 3 


3 cn 


3 £ 7 
D.D 1 


U.U_» 


D.HO, J. DO 


Frequency of events reported 
















Intervention 


2.93 


0.07 


2.80, 


3.06 


2.77 


0.06 


2.66, 2.89 


Control 


2.72 


0.05 


2.62, 


2.82 


2.80 


0.07 


2.67, 2.93 


Unit manager promoting safety 
















Intervention 


3.78 


0.06 


3.66, 


3.90 


3.70 


0.06 


3.56, 3.82 


Control 


3.56 


0.06 


3.44, 


3.67 


3.52 


0.07 


3.38, 3.65 


Organizational learning 
















Intervention 


3.43 


0.05 


3.34, 


3.53 


3.50 


0.05 


3.41, 3.60 


Control 


3.27 


0.05 


3.18, 


3.37 


3.33 


0.06 


3.22, 3.45 


Teamwork within units 
















Intervention 


3.66 


0.05 


3.55, 


3.76 


3.72 


0.05 


3.62, 3.81 


Control 


3 ci; 


n n/. 


3 /. c 


3 C3 


D.DH 


U.U_» 


'i I.I. 3 CI. 
D.HH, D.OH 


Communication openness 
















Intervention 


3.67 


0.05 


3.56, 


3.78 


3.61 


0.06 


3.50, 3.72 


Control 


3 C3 


n n/. 


3 /. 3 
J.4J, 


3 C1 


3 £ 7 


U.Ub 


J.4D, J.Oo 


Feedback/communication on error 
















Intervention 


3.33 


0.06 


3.20, 


3.45 


3.21 


0.06 


3.08, 3.33 


Control 


3.07 


n 


9 Qfl 


J.l 1 


9 Qfl 


n n£ 


Z.OJ, J.1U 


Non-punitive response to error 
















Intervention 


3.88 


0.05 


3.78, 


3.98 


3.89 


0.04 


3.80, 3.98 


Control 


3.68 


0.05 


3.57, 


3.78 


3.70 


0.06 


3.59, 3.82 


Adequate staffing 
















Intervention 


3.44 


0.05 


3.34, 


3.54 


3.58 


0.05 


3.48, 3.67 


Control 


3.35 


0.05 


3.26, 


3.45 


3.29 


0.06 


3.17, 3.40 


Hospital management promoting safety 














Intervention 


2.80 


0.06 


2.69, 


2.93 


2.90 


0.06 


2.78, 3.02 


Control 


2.86 


0.05 


2.76, 


2.96 


2.95 


0.06 


2.83, 3.07 


Teamwork across units 
















Intervention 


3.06 




9 Q7 

z.y /, 


31/ 

6. lH 


3 r\3 


a n/. 


9 Q/ 3 11 


Control 


3.08 


0.04 


3.00, 


3.15 


3.13 


0.04 


3.05, 3.21 


Quality of handoffs and transitions 
















Intervention 


3.03 


0.05 


2.93, 


3.12 


3.05 


0.05 


2.96, 3.15 


Control 


3.05 


0.04 


2.97, 


3.13 


3.17 


0.05 


3.08, 3.26 


Sum mean score (12 factors) 
















Intervention 


3.39 


0.03 


3.32, 


3.45 


3.39 


0.03 


3.32, 3.45 


Control 


3.27 


0.03 


3.20, 


3.33 


3.29 


0.04 


3.22, 3.36 



that poor organizational culture and deeper-running pro- 
blems can undermine the effectiveness of interventions 
like checklists. 9 Interestingly, the compliance rates with 
the SSC in this study were rather high (85%, 84%, and 
77% for the Sign In, Time Out, and Sign out phases, re- 
spectively) which indicate fairly successful early implemen- 
tation. Anecdotal evidence observed by and also relayed 
to the research team also concurred that there were no 
major problems. This compares favourably with findings 
from other countries like the UK pilot implementation of 
the SSC, which was met with some resistance and 



compliance ranged from 42% to 80%. 31 Strategies for suc- 
cessful implementation of the SSC have included education 
(training and materials), champions, organizational leader- 
ship, clear roles in the team, regular audits, feedback, and 
local adaptation 5 31 36 — which are all elements that we 
used during implementation. 

We thus have a rather paradoxical effect of a reasonably 
successful introduction of the SSC intervention but no 
major cultural impact. A number of explanations could be 
put forward here — all of which are amenable to further 
study. A first possibility is that culture and SSC are 
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Fig 2 WHO SSC impact on safety culture perceptions of 'frequency of events reported' (near misses in theatres) and 'adequate staffing' (to be 
able to handle any difficult situation in theatre), before and after the SSC intervention at Haukeland University Hospital, Bergen, Norway in 
2009-2010. CI, confidence interval. 



Table 4 Effects on safety culture factors of the intervention (checklist) compared with control groups and pre-intervention (n=349) vs 
post-intervention (n=292) survey phases estimated by the linear mixed model, tp,- (/=1, 2, 3), estimated regression coefficients; f> 0 . estimated 
mean/constant; CI, confidence interval; NS, not significant and not included; ^With interaction: y=p 0 +Pi-checklist group+(3 2 -post intervention 
survey+p 3 -checklist groupxpost intervention survey; without interaction: y=|3o+pi-checktist group+(3 2 -post intervention survey+0. *P<0.05; 
**P<0.01 



Predictors 




Constant 


Differences for group 
checklist vs control 


Overall change post- 
vs pre-intervention 
survey 


Checklist effect for 
group x survey (pre/ 
post) interaction 


Safety factors (scale 1-5) 


Items p 0 


95% CI 


Pi 


95% CI 


P2 


95% CI 


Ps 


95% CI 


Overall patient safety 


4 


3.49 


3.43, 4.57 


0.14 


0.03, 0.24* 


0.06 


-0.01, 0.13 


NS 




Frequency of events reported 


3 


2.71 


2.61, 2.81 


0.20 


0.04, 0.35* 


0.11 


-0.02, 0.23 


-0.25 


-0.43, -0.07** 


Unit manager promoting safety 


4 


3.54 


3.44, 3.64 


0.22 


0.08, 0.36** 


-0.05 


-0.13, 0.03 


NS 




Organizational learning 


3 


3.27 


3.19, 3.35 


0.16 


0.04, 0.27** 


0.07 


-0.00, 0.15 


NS 




Teamwork within units 


4 


3.54 


3.47, 3.62 


0.14 


0.03, 0.25** 


-0.01 


-0.08, 0.06 


NS 




Communication openness 


3 


3.52 


3.43, 3.60 


0.11 


-0.01, 0.23 


0.02 


-0.06, 0.09 


NS 




Feedback/communication on error 


3 


3.03 


2.97, 3.16 


0.24 


0.10, 0.37** 


-0.08 


-0.16, 0.01 


NS 




Non-punitive response to error 


3 


3.65 


3.57, 3.74 


0.21 


0.09, 0.33** 


0.01 


-0.07, 0.09 


NS 




Adequate staffing 


4 


3.33 


3.25, 3.42 


0.10 


-0.04, 0.23 


-0.07 


-0.17, 0.03 


0.21 


0.07, 0.35** 


Hospital management promoting 


3 


2.84 


2.75, 2.94 


-0.04 


-0.17, 0.10 


0.12 


0.04, 0.20** 


NS 




safety 




















Teamwork across units 


4 


3.09 


3.02, 3.15 


-0.05 


-0.14, 0.05 


0.03 


-0.03, 0.09 


NS 




Quality of handoffs and transitions 


4 


3.06 


2.98, 3.13 


-0.04 


-0.15, 0.07 


0.08 


0.01, 0.14* 


NS 




SUM score (mean) 


42 


3.26 


3.20, 3.31 


0.12 


0.04, 0.20** 


0.02 


-0.02, 0.06 


NS 





unrelated— but in the light of previous evidence, this is not 
the likeliest possibility. Secondly, the baseline culture levels of 
these services were already high— hence a ceiling effect pre- 
vented further improvements. A third, related possibility 
is that the timeline was too short to obtain such an 
improvement— after all introducing a new procedure is fairly 
quick, whereas a shift in experienced professionals' mind sets 
regarding their organization and practice might require a 
longer gestation period. Both of these explanations require 



longitudinal ongoing evaluations of culture and its fluctua- 
tions—and linking these with Checklist utilization. Cross- 
sectional studies between different countries and healthcare 
systems currently using the Checklist (e.g. Norway and 
UK) would also be useful in this respect. Further, observational 
assessments of how the SSC is actually used within the pressur- 
ized theatre environment are also required — culture measures 
are useful, but they cannot account for people reporting one 
thing yet doing another. A fourth explanation is that people 
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can change their behaviour without necessarily visibly chan- 
ging their underlying attitudes. Psychological theory suggests 
that this cannot hold for a very long time, as people strive 
to be consistent between their attitudes (i.e. perceptions of 
culture) and their behaviour 37 (i.e. usage of checklist)— which 
makes more compelling the longitudinal evaluation of both 
behaviour and culture perceptions. 

Limitations and strengths 

The response rate at baseline (61%) and at post intervention 
(51%) might be a limitation for sample representativeness. 
There were differences in professional backgrounds 
between responders and non-responders (but not for other 
patient characteristic factors). The significant differences 
within groups and variations within professions could indi- 
cate study weaknesses, thus the MLM analysis 30 adjusted 
for these— and indeed inclusion of the covariates in the ana- 
lyses did not influence the results. Finally, information about 
the Checklist intervention and local enthusiasm could have been 
transferred to individuals in the control group and biased results 
('spill-over' effect) — which is something that could not be con- 
trolled. In balance, key strengths of this study are the use of a 
carefully controlled design and matched assessments of safety 
culture pre- and post-intervention at the individual participant 
level (rather than group level). 

Implications 

Our findings, and overall experience with the study, have impli- 
cations for the introduction of safety interventions, like the SSC, 
and for further research on the effectiveness of such interven- 
tions. In this study, seven of the safety culture factors showed 
overall significant differences between the intervention and 
control groups, with the intervention group being significantly 
more positive. Following WHO advice, one could advocate 
that implementation of an intervention should begin with 
healthcare teams or professionals who are positive towards 
the intervention — hence the concept of 'champions'. This, 
however, might be a challenge when designing evaluation 
studies that include a control group, as entire units or operat- 
ing theatres that are more positive towards the intervention 
might show a 'ceiling effect' — that is, the size of the improve- 
ment triggered by the intervention is smaller in these groups 
precisely because they are more positively predisposed to 
the intervention to start with. Pre-/post-intervention designs 
are not the most suitable to tease out such effects — and 
indeed, we would argue that a deeper understanding of how 
exactly a healthcare organization moves across dimensions 
of culture over time cannot be gauged by such studies. We 
would thus advocate periodic and systematic assessments 
of an organization's culture using a well-validated instrument. 
This will allow longitudinal, time-series-based evaluation of 
whether the organization (or parts of it) moves in a certain dir- 
ection, and whether interventions are causing such shifts. 

Further, feedback of such measures within the organization 
can provide better self-insight and allow clinical units to self- 
evaluate and to compare themselves with their peers. We 
would hypothesize that such organization-wide assessments 



are an intervention in themselves and that a positive relation- 
ship should be expected between them and organizational 
readiness to improve safety and quality and to adopt novel 
interventions. These hypotheses await further study. 
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Appendix 

The aim of the MLM regression analysis is to describe the 
effect of the checklist intervention on primary outcome; the 
safety culture factors of the Hospital SOPS (dependent vari- 
ables) with intervention and control groups at baseline and 
post-intervention surveys (independent variables). 

A hierarchical model was formed for the interaction and 
expressed as: 

Y = p 0 + Pi x Group + p 2 x Survey + fi 3 

x Group x Survey (Al) 

where f) 0 is the model constant/intercept, fa 2 3 the regres- 
sion coefficients (estimated by restricted maximum likelihood 
methods), Group the independent variable as intervention 
(=1) or control (=0) group, and Survey the independent vari- 
able as baseline survey (=1) and post-intervention (=0) survey. 

The models assume covariance type CSR (compound sym- 
metry with correlation parameterization) for repeated response 
from subjects at baseline and at post-intervention. The ana- 
lyses also include respondents replying only at baseline or post- 
intervention. Independent variables were fixed. To adjust for 
co-variables, we included profession, gender, patient contact, 
and work experience in the hospital in the models: 

Y = p 0 + fa x Group + fa x Survey + fa 

x Profession H h fa x Group x Survey (A2) 

For safety culture factors without significant interaction effects 
of the checklist intervention, we used an equation for asses- 
sing the variations between groups and between surveys: 

Y = fa + fa x Group + fa x Survey (A3) 
Equations (Al) and (A3) are used in Table 4. 

Handling editor: H. C. Hemmings 
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